Learning Contextual Reward Expectations for Value Adaptation

Substantial evidence indicates that subjective value is adapted to the statistics of reward expected within a given temporal context. However, how these contextual expectations are learned is poorly understood. To examine such learning, we exploited a recent observation that participants performing a gambling task adjust their preferences as a function of context. We show that, in the absence of contextual cues providing reward information, an average reward expectation was learned from recent past experience. Learning dependent on contextual cues emerged when two contexts alternated at a fast rate, whereas both cue-independent and cue-dependent forms of learning were apparent when two contexts alternated at a slower rate. Motivated by these behavioral findings, we reanalyzed a previous fMRI data set to probe the neural substrates of learning contextual reward expectations. We observed a form of reward prediction error related to average reward such that, at option presentation, activity in ventral tegmental area/substantia nigra and ventral striatum correlated positively and negatively, respectively, with the actual and predicted value of options. Moreover, an inverse correlation between activity in ventral tegmental area/substantia nigra (but not striatum) and predicted option value was greater in participants showing enhanced choice adaptation to context. The findings help understanding the mechanisms underlying learning of contextual reward expectation

Supergravity Higgs Inflation and Shift Symmetry in Electroweak Theory

We present a model of inflation in a supergravity framework in the Einstein frame where the Higgs field of the next to minimal supersymmetric standard model (NMSSM) plays the role of the inflaton. Previous attempts which assumed non-minimal coupling to gravity failed due to a tachyonic instability of the singlet field during inflation. A canonical K\"{a}hler potential with \textit{minimal coupling} to gravity can resolve the tachyonic instability but runs into the $\eta$-problem. We suggest a model which is free of the $\eta$-problem due to an additional coupling in the K\"{a}hler potential which is allowed by the Standard Model gauge group. This induces directions in the potential which we call K-flat. For a certain value of the new coupling in the (N)MSSM, the K\"{a}hler potential is special, because it can be associated with a certain shift symmetry for the Higgs doublets, a generalization of the shift symmetry for singlets in earlier models. We find that K-flat direction has $H_u^0=-H_d^{0*}.$ This shift symmetry is broken by interactions coming from the superpotential and gauge fields. This direction fails to produce successful inflation in the MSSM but produces a viable model in the NMSSM. The model is specifically interesting in the Peccei-Quinn (PQ) limit of the NMSSM. In this limit the model can be confirmed or ruled-out not just by cosmic microwave background observations but also by axion searches.Comment: matches the published version at JCA

The Dopaminergic Midbrain Mediates an Effect of Average Reward on Pavlovian Vigor

Dopamine plays a key role in motivation. Phasic dopamine response reflects a reinforcement prediction error (RPE), whereas tonic dopamine activity is postulated to represent an average reward that mediates motivational vigor. However, it has been hard to find evidence concerning the neural encoding of average reward that is uncorrupted by influences of RPEs. We circumvented this difficulty in a novel visual search task where we measured participants' button pressing vigor in a context where information (underlying an RPE) about future average reward was provided well before the average reward itself. Despite no instrumental consequence, participants' pressing force increased for greater current average reward, consistent with a form of Pavlovian effect on motivational vigor. We recorded participants' brain activity during task performance with fMRI. Greater average reward was associated with enhanced activity in dopaminergic midbrain to a degree that correlated with the relationship between average reward and pressing vigor. Interestingly, an opposite pattern was observed in subgenual cingulate cortex, a region implicated in negative mood and motivational inhibition. These findings highlight a crucial role for dopaminergic midbrain in representing aspects of average reward and motivational vigor

Penggunaan Media Pembelajaran Pendidikan Kewarganegaraan untuk Meningkatkan Motivasi dan Hasil Belajar

The background research that the level of difficulty for students at SDN 14 Sadaniang in civic education absorbing subject matter is quite alarming.The purpose of this study to describe the use of instructional civics teaching media to improve motivation and learning outcomes in grade IV SDN 14 Sadaninag. The method used in this study was Quasi Experimental Research. These samples included 20 students. Data analysis techniques used t test. The research findings are that there are differences in student motivation and learning outcomes without the use of instructional media civics in class IV SDN 14 Sadaniang. The research findings are that motivation and student learning outcomes before using instructional media is low because of the three meetings, there are students who do not reach the KKM. While using the media after learning motivation and learning outcomes of students increased because of 3 sessions, all students reach KKM

Effects of Noise in a Cortical Neural Model

Recently Segev et al. (Phys. Rev. E 64,2001, Phys.Rev.Let. 88, 2002) made long-term observations of spontaneous activity of in-vitro cortical networks, which differ from predictions of current models in many features. In this paper we generalize the EI cortical model introduced in a previous paper (S.Scarpetta et al. Neural Comput. 14, 2002), including intrinsic white noise and analyzing effects of noise on the spontaneous activity of the nonlinear system, in order to account for the experimental results of Segev et al.. Analytically we can distinguish different regimes of activity, depending from the model parameters. Using analytical results as a guide line, we perform simulations of the nonlinear stochastic model in two different regimes, B and C. The Power Spectrum Density (PSD) of the activity and the Inter-Event-Interval (IEI) distributions are computed, and compared with experimental results. In regime B the network shows stochastic resonance phenomena and noise induces aperiodic collective synchronous oscillations that mimic experimental observations at 0.5 mM Ca concentration. In regime C the model shows spontaneous synchronous periodic activity that mimic activity observed at 1 mM Ca concentration and the PSD shows two peaks at the 1st and 2nd harmonics in agreement with experiments at 1 mM Ca. Moreover (due to intrinsic noise and nonlinear activation function effects) the PSD shows a broad band peak at low frequency. This feature, observed experimentally, does not find explanation in the previous models. Besides we identify parametric changes (namely increase of noise or decreasing of excitatory connections) that reproduces the fading of periodicity found experimentally at long times, and we identify a way to discriminate between those two possible effects measuring experimentally the low frequency PSD.Comment: 25 pages, 10 figures, to appear in Phys. Rev.

Role of the initial conditions on the enhancement of the escape time in static and fluctuating potentials

We present a study of the noise driven escape of an overdamped Brownian particle moving in a cubic potential profile with a metastable state. We analyze the role of the initial conditions of the particle on the enhancement of the average escape time as a function of the noise intensity for fixed and fluctuating potentials. We observe the noise enhanced stability effect for all the initial unstable states investigated. For a fixed potential we find a peculiar initial condition $x_c$ which separates the set of the initial unstable states in two regions: those which give rise to divergences from those which show nonmonotonic behavior of the average escape time. For fluctuating potential at this particular initial condition and for low noise intensity we find large fluctuations of the average escape time.Comment: 8 pages, 6 figures. Appeared in Physica A (2003

Targeted metabolomic analyses of cellular models of pelizaeus-merzbacher disease reveal plasmalogen and myo-inositol solute carrier dysfunction

<p>Abstract</p> <p>Background</p> <p>Leukodystrophies are devastating diseases characterized by dys- and hypo-myelination. While there are a number of histological and imaging studies of these disorders, there are limited biochemical data available. We undertook targeted lipidomic analyses of Pelizaeus-Merzbacher disease (PMD) fibroblasts, PMD lymphocytes, and 158JP oligodendrocytes, a murine model of PMD, to define the lipid changes in these cell models. Further targeted metabolomics analyses were conducted to obtain a preliminary evaluation of the metabolic consequences of lipid changes and gene mutations in these cell models.</p> <p>Results</p> <p>In both PMD fibroblasts and lymphocytes, and 158JP oligodendrocytes, ethanolamine plasmalogens were significantly decreased. Labeling studies with 158JP oligodendrocytes further demonstrated a decreased rate of lipid remodeling at sn-2. Targeted metabolomics analyses of these cells revealed dramatic increases in cellular levels of myo-inositol. Further uptake studies demonstrated increased rates of myo-inositol uptake by PMD lymphocytes.</p> <p>Conclusions</p> <p>Our data demonstrating PlsEtn decrements, support previous studies indicating leukodystrophy cells possess significant peroxisomal deficits. Our data for the first time also demonstrate that decrements in peroxisomal function coupled with the PLP1 gene defects of PMD, result in changes in the function of membrane myo-inositol solute carriers resulting in dramatic increases in cellular myo-inositol levels.</p

Combined model-free and model-sensitive reinforcement learning in non-human primates

Contemporary reinforcement learning (RL) theory suggests that potential choices can be evaluated by strategies that may or may not be sensitive to the computational structure of tasks. A paradigmatic model-free (MF) strategy simply repeats actions that have been rewarded in the past; by contrast, modelsensitive (MS) strategies exploit richer information associated with knowledge of task dynamics. MF and MS strategies should typically be combined, because they have complementary statistical and computational strengths; however, this tradeoff between MF/MS RL has mostly only been demonstrated in humans, often with only modest numbers of trials. We trained rhesus monkeys to perform a two-stage decision task designed to elicit and discriminate the use of MF and MS methods. A descriptive analysis of choice behaviour revealed directly that the structure of the task (of MS importance) and the reward history (of MF and MS importance) significantly influenced both choice and response vigour. A detailed, trial-by-trial computational analysis confirmed that choices were made according to a combination of strategies, with a dominant influence of a particular form of model sensitivity that persisted over weeks of testing. The residuals from this model necessitated development of a new combined RL model which incorporates a particular credit assignment weighting procedure. Finally, response vigor exhibited a subtly different collection ofMFand MS influences. These results provide new illumination onto RL behavioural processes in non-human primates

Deep brain stimulation of the subthalamic nucleus modulates sensitivity to decision outcome value in Parkinson's disease.

Deep brain stimulation (DBS) of the subthalamic nucleus in Parkinson's disease is known to cause a subtle but important adverse impact on behaviour, with impulsivity its most widely reported manifestation. However, precisely which computational components of the decision process are modulated is not fully understood. Here we probe a number of distinct subprocesses, including temporal discount, outcome utility, instrumental learning rate, instrumental outcome sensitivity, reward-loss trade-offs, and perseveration. We tested 22 Parkinson's Disease patients both on and off subthalamic nucleus deep brain stimulation (STN-DBS), while they performed an instrumental learning task involving financial rewards and losses, and an inter-temporal choice task for financial rewards. We found that instrumental learning performance was significantly worse following stimulation, due to modulation of instrumental outcome sensitivity. Specifically, patients became less sensitive to decision values for both rewards and losses, but without any change to the learning rate or reward-loss trade-offs. However, we found no evidence that DBS modulated different components of temporal impulsivity. In conclusion, our results implicate the subthalamic nucleus in a modulation of outcome value in experience-based learning and decision-making in Parkinson's disease, suggesting a more pervasive role of the subthalamic nucleus in the control of human decision-making than previously thought.GRF gratefully acknowledges support by the German Research Council (Deutsche Forschungsgemeinschaft, KFO-219). Ray Dolan is supported by the Wellcome Trust (R.J.D., Senior Investigator Award 098362/Z/12/Z) and the the Senate of Berlin (R.J.D., Einstein Fellowship). Ben Seymour is funded by the Wellcome Trust and the National Institute of Information and Communications Technology, Japan; Peter Dayan is funded by the Gatsby Charitable Foundation

The effect of motivation on movement: a study of bradykinesia in Parkinson's disease.

Bradykinesia is a cardinal feature of Parkinson's disease (PD). Despite its disabling impact, the precise cause of this symptom remains elusive. Recent thinking suggests that bradykinesia may be more than simply a manifestation of motor slowness, and may in part reflect a specific deficit in the operation of motivational vigour in the striatum. In this paper we test the hypothesis that movement time in PD can be modulated by the specific nature of the motivational salience of possible action-outcomes